As a key technique in the video streaming framework of H. 264 and MPEG-4, Fine Granularity Scalability (FGS) video coding is able to implement arbitrary truncation of the video code stream by post-processing after coding. The code stream generated by the FGS technique may be adapted to fluctuations of the network bandwidth.
In FGS video coding, motion compensation may improve the coding efficiency and decrease bandwidth requirement. The method for implementing motion compensation is to subtract the reconstructed frame of the motion-compensated encoded frame from the current frame so that the difference between the current frame and the reconstructed frame is encoded for transmission whereas the unchanged content in the current frame remains uncoded and untransmitted. In this way, a motion value can be obtained by estimating the movement of the content of the previous frame, and a higher compression ratio can be achieved by compensating with such motion value in the current frame. The motion compensation is conducted in a closed prediction loop. In the encoding end, the difference between the original signal and the prediction signal after the prediction is transformed on a block basis. The transformation coefficient forms an output code stream after quantization and coding. In the decoding end, the quantized transformation coefficient may be inversely transformed and subsequently added to the current prediction signal to form a reconstructed frame which, in turn, serves as the prediction signal for a successive sequence.
The code stream of the FGS video coding includes a base layer and at least one enhancement layer. In order to improve the coding efficiency, a high-quality reference image, constituted by part of the enhancement layer, is adopted during enhancement layer coding. In the case where the network is inadequate to transmit the code stream of the enhancement layer due to the reasons, such as network fluctuation, the decoding end will abandon parts of or the entire high-quality reference image. Such difference of the high-quality images between the encoding end and the decoding end inevitably leads to error propagation and accumulation and thus impairs the quality of the encoded image. To overcome the foregoing defects, a leaky factor α is introduced in the enhancement layer so as to perform an additional motion compensation upon the reconstructed image of high quality. Motion compensation with leaky factor α means that, when reconstructing the high-quality reference image, α attenuation is initially performed upon the coefficients obtained by subtracting the reconstructed base layer from the high-quality prediction image of the previous frame, and then the attenuated coefficients are added with the reconstructed base layer to constitute a high-quality reference image of motion compensation. The quality of the reference image can be controlled by selecting different values for a so as to make the quality of the reconstructed frame allowable by the network bandwidth as best as possible and minimize the difference between the image of the encoding end and the image of the decoding end. Currently, a global α is generally set on the basis of the entire image sequence. The way to set the global leaky factor α is to encode a leaky factor in every piece of slice header information and then change the probability of the amount of leakage (i.e., adjust the leaky factor) by the decoder according to the state of the current frame. This method is a trade-off between the prevention of prediction drift and the compression of efficiency.
Generally, in practice, the above method of determining the global leaky factor α can be employed when the following requirements are satisfied. The above method may be employed when there is no transformation coefficient to be encoded in the base layer, for example, in the instance where the prediction error for the amount of the drift is very small. In this case, the prediction of the amount of drift already has a sound performance. Alternatively, the above method may be employed when there are few transformation coefficients of importance in the base layer. In this case, the coefficients to which the leakage prediction for the enhancement layer corresponds may be excluded. The adjustment of the leaky factor is accomplished by computing every signal transformation and inverse transformation in the prediction loop. In the process of adjusting the leaky factor, the leaky factor needs to be selectively adjusted in the transformation domain. Alternatively, the above method of determining the global leaky factor may be employed when there are quite a lot of transformation coefficients of importance in the base layer and the leakage prediction is completely closed.
The decoding end in the method adjusts the leaky factor according to the current frame and the drift. However, the actual drift is not generated by the current frame. Therefore, it is not ideal to utilize the leaky factor which is set in accordance with the above method to control the amount of drift, which means that the above method of setting the leaky factor is not able to set a best leaky factor. Moreover, in the above method of setting the leaky factor, additional transformation and inverse transformation has to be introduced such that the complexity of encoding and decoding computation is increased. Therefore, the method may not be universally applicable.